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Abgt r&Qt : Six professionally active, retired captains 
rated the coordination and decisionmaking performances 
of sixteen aircrews while viewing videotapes of a simu- 
lated commercial air transport operation. The video- 
tapes displayed a composite of four views of 
crewmembers, and the cockpit, from cameras located 
inside the simulator . The scenario featured a required 
diversion and a probable minimum fuel situation. Seven 
point Likert-type scales were used in rating variables 
on the basis of a model of crew coordination and 
decisionmaking. The variables were based on concepts 
of, for example, decision difficulty, efficiency, and 
outcome quality; and leader-subordinate concepts such 
as person- and task-oriented leader behavior , and com- 
petency motivation of subordinate crewmembers. Five- 
front-end variables of the model were in turn dependent 
variables for a hierarchical regression procedure. The 
variance in safety performance was explained 46%, by 
decision efficiency, command reversal, and decision 
quality. The variance of decision quality, an alterna- 
tive substantive dependent variable to safety perfor- 
mance, was explained 60% by decision efficiency and the 
captain's quality of within-crew communications . The 
variance of decision efficiency, crew coordination, and 
command reversal were in turn explained 78%, 80%, and 
60% by small numbers of preceding independent vari- 
ables. A principle component, varimax factor analysis 
supported the model structure suggested by regression 
analyses. Crewmembers for this study were diverse with 
respect to airline of origin and recency, or currency 
on the Boeing 707 - the aircraft simulated. Some 
retired personnel were used. The results should be 
interpreted accordingly. 


INTRODUCTION 
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The aircrew interaction process has been implicated as 
contributing to numerous recent air transport accidents and 
incidents (Cooper, White, & Lauber, 1979; Murphy, 1980; NTSB, 
1976) . And many interpersonal factors have been suggested as 
causes of ineffective crew performance {lack of decisive 
command, strained social relations, and pilot-copilot role 
relationships (Murphy, 1977)}. Problematic pilot-copilot 
role issues include the command responsibility of the cap- 
tain when the first officer is flying, and the responsibil- 
ity of the first officer when the captain deviates from safe 
or legal practices (Wiener, 1977). 

Flightcrew communications patterns have been related to 
performance outcomes in a study of simulator data (Foushee 
and Manos , 1981). Mitigation level, a linguistic indication 
of tentativeness and indirectness in speech, has been iden- 
tified as a factor in failures of crewmembers to get new 
topics discussed or suggestions ratified by the captain 
(Goguen, Linde, and Murphy, 1984). In their study of air 
transport accident transcripts they also showed mitigation 
level to vary with command and situation dimensions. 
Finally, a full mission simulator study of crew performance 
(Ruffell-Smith, 1979) related ineffective management of both 
human and material resources to increased decision times. 
Generally, however, suggested causal factors in air crew 
performance effectiveness have not been well defined through 
systematic study or research. One reason for this could be 
the lack of adequate methods for isolating and quantifying 
crew interaction factors and for relating these factors to 
flight task performance (Foushee, 1984; Murphy, 1977) - a 
situation comparable to that for small group performance 
generally (Hackman and Morris, 1975). 

The major objective of this rating study was to ini- 
tiate development of a hierarchical process model of aircrew 
coordination and decisionmaking. A secondary objective was 
to develop reliable measures of the crew interaction process 
that could be related to other substantive measures, such as 
flight task error measures, or to measures developed with 
coded communications data. This will be addressed in future 
reports . 

This study used videotapes of aircrews performing a 
full mission simulation of a commercial air transport opera- 
tion (Murphy, Randle, Tanner, Frankel, Goguen, and Linde, 
1984) . Such videotapes have been used in studying medical 
team-patient interactions (Frankel & Beckman, 1982). Leader- 
ship style and crewmember competency variables, included in 
the model of crew performance presented below, reflect find- 
ings from recent critical reviews of the leadership litera- 
ture (House, 1984; House & Baetz, 1979; Kerr, 1984). Design 
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of the rating scales and procedures (including rater train- 
ing), reflect findings from recent reviews of rating litera- 
ture (Landy & Farr. 1983; Landy & Farr, 1980). The single 
dimension , Likert-type scales and anchoring methods were 
justified on the bases that the study is exploratory, and 
research findings have shown little gain in performance when 
more complex scales are used. 


METHOD 


The MQdel 

Crewmem be r Behavioral Variables: Variables of major 
interest were based on some focal concepts. Task-oriented, 
person-oriented, and participatory leadership behaviors were 
rated for both the captain and first officer. These vari- 
ables were differentiated on the basis of specific 
behaviors. Task-oriented leadership behaviors were those 
concerned with establishing goals, clarifying responsibili- 
ties, defining subordinate (others for the first officer) 
roles and task requirements, coaching subordinates and pro- 
viding task related feedback. Person-oriented leadership 
behaviors included those evidencing concern with establish- 
ing and maintaining positive crewmember relationships, pro- 
viding psychological support, and enabling feelings of 
satisfaction. 

Participatory leadership was rated in regard to 
supervision/ resources management on the basis of behavior 
that encouraged subordinates (or other crewmembers, for 
first officer) to make suggestions regarding accomplishment 
of tasks, independently analyze problems, give feedback, and 
question the leader. Participatory leadership was also rated 
in regard to decisionmaking. The criterion was behavior con- 
cerned with ensuring that all crewmembers for whom a deci- 
sion was relevant had a chance to influence that decision. 
Relevance was indicated if a crewmember had significant and 
pertinent information related to the decision, responsibil- 
ity for implementing the decision, or significant ego 
involvement for other reasons . 

To address the question of interaction between a 
captain ' s leadership effectiveness and a subordinate ' s capa- 
city and willingness to participate , the first officer and 
flight engineer were rated on a dimension of competency- 
motivation . The criterion was evidence of a crewmember being 
knowledgeable , skillful , and motivated with respect to ful- 
filling the requirements of his position . 
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Within-crew communications quality was rated, for all 
three crewmembers with respect to both specific decisionmak- 
ing processes and participation in the more general crew 
coordination process. Behavioral criteria were 1) hearable, 
understandable, appropriate (in style and content), accu- 
rate, and timely messages; 2) being a good listener who 
makes an effort to understand; and 3) achieving a reciprocal 
indication that understanding was reached. 

Command ReYOrg&l - An In terac tive Var ia b le : Command 
reversal was also rated with respect to both specific 
decisionmaking processes and the more general crew process. 
If, for example, a first officer performed much of what 
would normally be the captain's general leadership function, 
a high rating for that crew on the variable "command rever- 
sal" would be expected. Similarly, if a first officer per- 
formed much of what would generally be the captain's 
decisionmaking function, a high rating for that crew on 
"decision command reversal" would be expected. No attempt 
was made to distinguish whether such command reversals were 
due to acquiescence of the captain or dominance of the first 
officer or whether the actions and decisions were or were 
not appropriate. 

Intermediate Per fo rma nce Me as ur es : Command reversal was 
expected to negatively affect crew coordination and decision 
efficiency, two other focal variables of the hypothesized 
hierarchical process model (shown in part in Figure 1). A 
high rating on crew coordination would indicate strong rater 
agreement that , over a mission segment : individual 
crewmember knowledge and skills were allocated in an effec- 
tive and timely manner to meet task and situation demands. A 
high rating on decision efficiency would indicate strong 
rater agreement that, for a particular decision process all 
significant information was acquired at an opportune time, 
adequately evaluated, and appropriately utilized. 

Depende nt Variab le s : Decision quality and safety per- 
formance, also shown in Figure 1, are alternative, primary 
dependent variables for this study. Like decision efficiency 
and some other variables mentioned above, decision quality 
was rated for eight decision processes that oocured during 
the mission. A high rating on decision quality would indi- 
cate strong rater agreement that a choice made was "the best 
considering safety of flight and/or the attainment of all 
other mission goals". Unlike the 15 crew process and nine 
decisionmaking variables, safety performance was rated on 
the basis of relatively factual data after raters had 
observed all 16 crews. Safety performance ratings were 
essentially based on an assessment of risk in a crew's solu- 
tion or attempted solution to the major scenario problem. 
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Data entering into safety performance included the airport 
where landing occured, fuel on board at landing, and alti- 
tudes reached during approaches below minimums when the run- 
way could not be seen. 

Other Variables : In addition to the focal variables 
discussed above, three variables of more peripheral interest 
were rated: crew cohesiveness, crew friendliness, and deci- 
sion difficulty. 

Identification an d De finitio ns e£ Va r iables : All vari- 
ables are identified with their concepts, referents, and 
instrument by which they were measured, in Table 1. Attach- 
ing the prefix "decision" to a concept such as participatory 
leadership distinguishes a variable referring to participa- 
tory behavior in decisionmaking as opposed to that in 
supervision/resources management. Attaching the suffix (PI), 
(P2) , or (P3) to a concept distinguishes a behavioral vari- 
able, such as communications quality, as to whether refer- 
ence is to captain, first officer, or flight engineer 
behavior, respectively. Definitions for all variables can be 
synthesized from the scale and criterion statements of 
Appendix A, presented so as to mirror concept presentation 
in Table 1 . 

The model will be further discussed below - including 
those assumptions leading to the partial formulation shown 
in Figure 1 . 

The Data 

The primary data for the study were sixteen high qual- 
ity, quad image tapes showing interaction and performance of 
sixteen three-man flight crews. The crews flew a full mis- 
sion scenario in a Boeing 720B flight training simulator, a 
late version of the Boeing 707. Figure 2 shows a typical 
quad image videotape frame: captain and first officer (upper 
left and right quadrants, respectively); flight engineer 
(lower right quadrant); and a context image shot from the 
back of the simulator that preserved the same relative loca- 
tions of crewmembers (lower left quadrant). This combined 
view was made from four small video cameras located in the 
simulator, out of sight of the crewmembers. 

A current, professional air traffic controller was used 
in the simulation. The controller also participated with 
another member of the experimental team in simulating 
conversations with other aircraft, to provide background 
conversations on the Air Traffic Control (ATC) network. 
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Crewme mb ers : The crewmembers were paid volunteers . 
Their experience represented a wide range of airline of ori- 
gin and recency, or currency on 33-707 line operations . Some 
were current on the B-707 . Many had recent B-707 line 
experience but were currently flying other jet aircraft in 
line operations . Some were retired from the line . Thus crew 
composition ranged from one in which all members were 
retired from the line to one currently flying the B-707 as 
an intact crew . The major objective of the overall simula- 
tion study was to develop methods for quantifying crew coor- 
dination and decisionmaking factors , and their relationships 
to flight task performance . Thus , this diversity in experi- 
ence was considered of some importance as an aid in evaluat- 
ing the sensitivity of candidate performance measures . 

All crewmembers received six hours of classroom differ- 
ences training and four to eight hours of simulator differ- 
ences training . The number of hours of simulator differences 
training that a crewmember received was based on recency. 
Subjects were formed into crews prior to simulator training 
and were instructed in coordinated procedures during this 
training . 

Sc en ario : Simply , the overall scenario represented a 
flight from Tuscon, continuing to Los Angeles (LAX) after a 
short stopover at Phoenix, with a forced diversion to an 
alternate upon reaching LAX . Each crew flew the scenario 
only once , without prior knowledge of the scenario problem. 
The intent of this procedure was to maximize a valid 
description of natural crew performance . The crew's enact- 
ment of the scenario began with a Captain ' s Briefing in the 
simulated operations room at Tuscon and ended upon stopping 
on the runway at the selected alternate { either Palmdale 
( PMD) or Ontario ( ONT ) } . This rating study used videotapes 
from the longer , problem-leg only - beginning as all three 
crewmembers entered the cockpit at Phoenix. 

The scenario was designed to evoke a series of deci- 
sions about where to proceed following a missed approach at 
LAX due to nose gear not-down-and-locked indication . This 
situation was exacerbated because it ocoured at a time when 
the Los Angeles basin (which includes the planned alternates 
Ontario and Long Beach) was experiencing low and deteriorat- 
ing ceiling and visibilities due to coastal fog . Following 
the missed approach and upon going through a complete gear 
check procedure that takes several minutes , the crews had to 
insure that the gear was down and pinned so they could 
assume that the panel light indication was faulty . 

Eight Deci si ons : While on the ground at Phoenix the 
crew was given weather information indicating some 
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degradation at LAX. During the latter part of the cruise to 
LAX, they would he given direct information and other cues 
about further deterioration of ceiling and runway visual 
range (RVR) at LAX. Cues included being given an enroute 
hold due to traffic back-up and an ATC-net conversation 
regarding another aircraft's missed approach and return for 
a second attempt. If weather conditions at possible alter- 
nates were requested a crew could realize that conditions at 
other coastal airports such as Long Beach were similar to 
LAX; that ONT located inland from LAX, was lagging LAX in 
deterioration; and that PMD, located just over a mountain 
range out of the Los Angeles basin, was experiencing clear 
weather with good visibility. The decisionmaking behavior of 
the crews with respect to whether unusual contingency plan- 
ning was required and what that planning should be was the 
first of eight decision processes to be rated in this Crews 
were cleared to approach LAX, from hold, when their fuel 
remaining was 14,000 lbs. This decision process was 

evaluated at the point of calling for gear down, a short 
time thereafter and near the outer marker at LAX. 

The other decisions concerned: 2) whether to go around 
on the first approach to LAX - a decision that is essen- 
tially procedural in that attempts to recycle the gear did 
not extinguish the failure indication; 3) whether to reap- 
proach LAX - some crews chose to proceed to an alternate and 
work the gear problem enroute 4) whether to go around 
(including early interruption) on the second approach to LAX 
somewhat less prooedurally based, depending on fuel 
remaining, for example; 5) the choice of ONT or PMD as an 
alternate; 6) whether to bring the nose gear up during 
cruise for fuel conservation; 7) whether to select another 
alternate after receiving company information on relative 
weather conditions at PMD and ONT and the companies' prefer- 
ence for ONT for passenger handling; and 8) what arrival 
status to declare. 

The last decision, like the first, is a complex deci- 
sion or planning process, and has components involving 
whether to declare an emergency or problem situation (due to 
the nose gear indication or for low fuel) and whether to 
request emergency equipment (if an emergency is not 
declared). The timing of information given by the company 
for decision process seven was designed to require crews to 
reconsider their alternate, but to maintain their original 
decision for a prudent outcome. 

The Rating Procedu re 

Rating Sc ales and A dmin istra ti on : Figure 3 shows the 
eight videotape stop points at which decision processes were 
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rated. At each point, raters marked nine decisionmaking (DM) 
scales like that presented in figure 4 - each occupying one 
page of a booklet . A criterion statement shown below a 
scale defines the underlined modifier in the scale statement 
at the top. Thus a very efficient decision process (see Fig- 
ure 4) is one in which "all significant information was 
acquired at an opportune time, adequately evaluated and 
appropriately utilized." 

The criterion statements tended to anchor scales at the 
top (?) scale value. During interactive rater training the 
kinds of outcomes that would merit rating at the other 
extreme (1) value and/or intermediate values, were dis- 
cussed. A blackboard beside the video playback unit con- 
tained complete rater instructions and a large scale with 
anchor descriptions for scale numbers two through six: neu- 
trality for four and incrementally equal interval tendencies 
toward strong agreement or disagreement for the others. 
Except for decision difficulty and decision quality scales, 
presented in that order at the beginning of each booklet, 
scales were presented in different random orders for each of 
the eight decisions. 

Figure 3 also shows that stop points one, five, and 
eight and the point at which all three crewmembers had 
entered the cockpit, defined the boundaries of three mission 
segments: 1) pre-problem, 2) major problem, and 3) secondary 
problem. At these three stop points, following administra- 
tion of the DM instrument, the 15-soale crew process (CP) 
instrument was administered. As contrasted to the DM instru- 
ment the CP instrument assessed qualities based on behavior 
of individual crewmembers, or the crew, throughout each seg- 
ment . Examples are task- and person- oriented leadership 
qualities and crew coordination. The 15 scales were 
presented in different random orders for each segment . 

As noted previously , the first videotape stop was made 
as "gear-down" was called at LAX. The eighth stop was made 
over the outer marker at the alternate. The other stops 
were keyed to completions of decision processes - usually 
signaled by the start of implementation. After stop point 
eight , the videotape was continued until the aircraft had 
stopped on the runway. At this time a fourth CP booklet was 
administered. These ratings on CP variables over the com- 
plete operation were made for comparison with average rat- 
ings over the three segments. 

All 25 scales were identical to that shown in Figure 3 
except for those that assessed first officer leadership 
styles. These contained a n/a position after number seven. 
N/a (not applicable) was to be circled only if no 
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opportunity arose for leader behavior. 

As has been noted, the safety performance (SP) instru- 
ment was administered in a session after raters had com- 
pleted all other ratings on the sixteen crews . The raters 
were not told that they would provide the SP ratings until 
the other rating sessions were completed . 

Airport , airway , and simulator performance information 
was available to raters during the rating process . Calcu- 
lated fuel requirements under emergency flight conditions , 
for both ideal and less-than-ideal aircraft configurations 
(e.g. gear down) were made available during rating of safety 
performance - f or go-around at ONT and PMD and for flights 
between airports . During all rating sessions , the videotape 
would be stopped at a rater ' s request to clarify a 
crewmember utterance or other factual information. The 
raters took notes throughout the flights , and were particu- 
larly encouraged to do so during the cruise from Phoenix to 
LAX. A monitor was present during all rating sessions to 
insure independence of ratings . 

Scenario conditions ( or contextual events) were not 
entirely consistent over the 16 crews . These inconsistencies 
as well as how they were dealt with are discussed in (Appen- 
dix B) . 

Raters: The raters were six retired captains , all main- 
taining professional experience as analysts or researchers 
with the NASA-Aviation Safety Reporting System . All had 
experience on the Boeing 720B and/or 707. Their combined 
airline experience totaled 224 years . Four airlines were 
represented. Year of retirement ranged from 1978 to 1984 . 

Eating Design: The raters were formed into two balanced 
groups of three raters , an A- and B-group , based on ASRS 
research experience , airline of experience , and recency of 
retirement . Two raters who worked in proximity to each other 
at their ASRS position were assigned to different groups 
all agreed not to discuss completed ratings with members of 
the other group . 

The A-group rated the videotapes , and hence crews , in 
order one through 16 . This crew identification order 
represented a randomization of the order that crews per- 
formed in the simulator . On each rating day , A-group rated 
two crews - one in a morning and one in the afternoon. The 
B-group rated crews in the general order of nine through 16 
followed by one through eight - except that morning and 
afternoon videotapes were reversed . Their actual order was: 
10,9,12,11. . . . The latin square type design provided control 
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for time of day effects , partial control for unanticipated 
sequential effects , and the possibility of examining data 
for such effects. 

Eatgr Training : All raters received three 2-hour ini- 
tial training sessions . In session one , the DM and CP rating 
booklets were presented and discussed . Feedback on these 
instruments was solicited and utilized when appropriate . A 
lecture was also given on rating theory - discussing , for 
example , assumptions of multidimensionality of jobs and 
situations ; effectiveness levels , or degrees of qualities 
within dimensions ; the rater as a measuring instrument ; rat- 
ing accuracy; need to reduce errors of halo , leniency, and 
midpoint cluster ; rating skill components ; and the desired 
end result of independent but reliable ratings . During ses- 
sions two and three accuracy and error reduction discussions 
were repeated . Also , DM and CP scales were utilized repeat- 
edly on videotapes made during " shakedown" simulator runs . 
These were non-data runs made by crews that were not 
included in the study . Following each rating effort , ratings 
were posted and discussed . A-group received an added train- 
ing session prior to starting ratings of crews nine through 
16 due to an unplanned 1-week interruption of their rating 
activity . 


RESULTS AND CONCLUSIONS 
Inter-rater Reliabilit ies 

Variable reliabilities were computed by use of the 
Spearman-Brown formula for multiple raters. These multiple 
r reliabilities are presented in Table 2. The average relia- 
bility over all variables is .92. 

Variable Means an d S ta ndard D ev iati on s 

Crew Process Variables: Means and standard deviations 
were computed for the 15 crew process variables for each of 
the three segments. The means for each variable were com- 
pared with Fisher's "protected t" test. Differences between 
segments 2 and 3 means were not significant (p >.10). 
Differences were significant (p <.05) for all variables - 
except competency-motivation (for P2 and P3), command rever- 
sal, and task-oriented leadership (P2) - between the pre- 
problem segment and each of the problem segments (see Table 
3). Thus, performance ratings declined on five of the six 
leadership behavior variables, on communications quality for 
all crewmembers, and on the three crew referenced variables 
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(cohesiveness, coordination, and friendliness) as the diffi- 
culty of the scenario was increased. These effects, 
apparently due to being in a less structured problem situa- 
tion, could have implication for remedial training. Table 3 
also presents the overall mean and SD for each of the 15 
crew process variables. An identical "t" test procedure 
also revealed no differences between an average rating on CP 
variables over the three segments and the overall rating 
made on the ground at the alternate. 

The standard deviations for the crew process variables 
across segments were fairly consistent within and between 
variables. The range of standard deviations within each of 
these variables across the three segments was approximately 
equal to the average range of .13. 

Decisionmaking Vari ab les : Means for each of the nine 
decision variables are presented for each of the eight deci- 
sions in Table 4. The overall mean and standard deviation 
for each variable is also presented in Table 4. 

The standard deviations for the decisionmaking vari- 
ables across decision points were fairly consistent within 
and between variables. The range of standard deviations 
across the eight decision points within each of these vari- 
ables was approximately equal to the average range of .41. 
Variables 16 and 17 had larger ranges (.72 and 1.01, respec- 
tively) . Inspection revealed that these larger ranges were 
due to the low standard deviations at decision point two 
(the first go-around) for these two variables. 

Safe ty Perf orm ance : The overall mean for safety perfor- 
mance ( V25 ) was 3.53, and the overall standard deviation was 
1.85. 

Multiple Regression /Correlation Analyses 

Corr el ation Ma tr ix : An interpair correlation matrix was 
obtained (n=96; 6 raters x 16 crews) (Table 5). The six 
raters made independent ratings. However, in that each rater 
rated each of the 16 crews, crews cannot be considered 
truly independent. Rather, there is a relative independence 
among the 96 points and some bias due to non-independence 
had to be accepted. The correlations of Table 5 that are at 
or above .267 are significant (p < . 01 ) for 90 degrees of 
freedom . 

The correlations between variables representing con- 
cepts assessed for both crew process and and decisionmaking 
are of interest for methodological reasons. Command rever- 
sal ( V9) is seen to correlate . 92 with decision command 
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reversal (V24). The possibility of combining these two 
variables for modeling purposes is suggested. The communi- 
cations quality variables ( V13-15) correlated .79, .73, and 

.78 with their decisionmaking counterparts (V21-23). The 
participatory leadership variables (V5,V6) correlated .83 
and .77 with their decisionmaking counterparts ( V19 , V20) . 

Crew Cohesiveness (V10) and Crew Coordination (VI 1) are 
significantly correlated (r=.93). There was evidence that a 
few raters had some difficulty in distinguishing these two 
variables conceptually - and it is easy to conceive of dif- 
ficulty in distinguishing them operationally. The expressed 
problem was in separating crew cohesiveness conceptually 
from crew coordination - not in rating crew coordination . 
For the prior reasons the high correlation may be , in part , 
an artifact . 

Regre ss ion Analyses: The partial model in Figure 1 is 
based on a set of assumptions that would determine the order 
for entering variables into a hierarchical regression 
analysis . These assumptions include the usual assumptions 
for establishing causal priority . Also , based on the 
hierarchical command structure it is assumed that oaptain 
leadership and communications qualities would have larger 
effects than those of other crewmembers . This assumption 
accounts for captain quality variables , and not other 
crewmember variables , being included in the front-end, par- 
tial model . The rationale for including a task-oriented 
leadership variable rather than person-oriented or partici- 
patory leadership variables was derived from some evidence 
that a task-oriented leadership style is more effective in 
problem situations . The curved line between V13 and V21 of 
Figure 1 indicates correlation but implies no causal rela- 
tionship , as the directed , signed lines do . 

The analytic approach chosen was a hierarchical pro- 
cedure initiated by a series of stepwise regressions , each 
subsequent procedure including decreasing numbers of the 
variables shown in Figure 1 . That is , all of the variables 
except V13 { communications quality ( PI ) } , which was included 
only in the last of the five regressions in the series . This 
was done to reduce the ratio of the k ( independent variables 
IV ' s ) to the n (96) for the first four regressions . 
Although these k/n ratios exceed what may be considered pru- 
dent for substantive findings , the exploratory nature and 
predominately predictive interest here , as well as use of an 
a priori hierarchical model for entry of initial variables 
is argued to justify the procedure . Through this procedure , 
k is restricted to small values relative to the large number 
of possible IVs . 
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The series of five stepwise regressions were performed 
in reverse order to that shown in Table 6. Table 6 indicates 
the dependent variables for these regressions as I) command 
reversal , II) crew coordination , III ) decision efficiency, 
IV) decision quality, and V) safety performance. 

Variables in the partial model that precede a particu- 
lar dependent variable were the IVs for that regression. 
Entering order for the IVs were top-down from left to right. 
The stepwise regression program employed was BMDP2R (BMDP 
Statistical Software, 1983) with a minimum acceptable F 
value (to enter) of 4 . 00 (p < . 05) and a maximum acceptable 
F value (to remove) of 3.90. 

Following each of the five basic regression procedures, 
the partial correlation table for variables not in the equa- 
tion was consulted to determine which F value to enter above 
4, if any . If such a variable was present and logically 
prior to the dependent variable , another regression, adding 
this variable , was performed . This procedure was continued 
until no logically prior variables had an F value above 4 to 
enter . 

Significant IVs were then entered into a final regres- 
sion by a usual hierarchical procedure - if precedences were 
strictly established by the model . If not , some alternative 
paths were usually considered . Regression results are dis- 
cussed in the order performed . Summary analyses are 
presented in Tables 6 and 7 . 

S afet y Pe rforma nce is seen (Table 7) to have 46% of the 
variance explained . Command reversal - a negatively corre- 
lated IV contributes 16%. Decision efficiency contributes 
28%. Decision quality contributes 3%. 

That decision quality only increments the variance 
explained by 3% could be due to two considerations . First , 
the definition of decision efficiency stops not too far 
short of including decision quality . The second and perhaps 
most important consideration is based on defining criteria 
for decision quality versus safety performance and is dis- 
cussed below . 

Decision Quality, considered an alternative substantive 
dependent variable to safety performance , is seen (Table 7) 
to have 60% of the variance explained by communications 
quality (PI) (12%) and decision efficiency (48%) . 

The suggested rationale for the relative percentages of 
variance explained for decision quality and safety perfor- 
mance is based on defining criteria for the two variables . 
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Defining criteria for decision quality were presented at the 
beginning of the rating effort and considered whether 
choices were most appropriate based on "safety of flight 
and/or the attainment of other mission goals." Safety per- 
formance ratings were based on the level of safety achieved 
(or risk avoided) in the end solution for the major scenario 
problem - or in unsafe attempts to solve the problem by 
landing at LAX - and criteria were presented only after all 
the other ratings had been made. Safety performance defining 
criteria thus excluded any consideration of attaining " other 
mission goals " - for example that ONT was preferable for 
passenger handling and was the designated alternate . 

Decision Efficiency is seen (Table 7) to have 78% of 
the variance explained. The explanatory variables and sug- 
gested increments in variance explained are decision commun- 
ications quality (P3) (17%), decision communications quality 
( PI ) (33%), decision command reversal (8%), command reversal 
(5%), and crew coordination (14%). 

This is the first stepwise procedure to be continued 
beyond the basic regression . Table 6 shows that , prior to 
these continuations 74% of the variance in decision effi- 
ciency was explained by the three variables of the formal 
partial model - decision communication quality (PI) , crew 
coordination and decision command reversal. The added vari- 
ables for the entering orders shown , contributed 2% incre- 
ments of variance respectively . Table 7 however shows that 
if the flight engineer's decision communications quality 
were considered logically prior to the other significant 
variables , it contributes 17% of the variance . The flight 
engineer ' s communications concerning the nose gear and fuel 
is suggested to explain the significance of this variable 
for decision efficiency . 

For most purposes , as will be further discussed below, 
V9 (command reversal) and V24 (decision command reversal) 
can be considered to be synonymous - or V24 can be used to 
assess the generic concept of command reversal , as the model 
of Figure 1 suggests . In explaining decision efficiency how- 
ever , V9 increments the variance explained by 5%. Decision 
command reversal is shown logically prior to command rever- 
sal in Figure 7 on the basis of the substantive dependent 
variables being decision based. 

Crew coordination is seen by Table 7 to have 81% of the 
variance explained . The order of explanatory variables and 
suggested increments in variance explained is essentially 
arbitrary . Perhaps the major implication from the regres- 
sions in Tables 6 and 7 is that crew coordination has most 
of its variance explained by a quality variable for each of 


23.14 



the three crewmembers: Communications quality (PI), 
competency-motivation (P2), and competency-motivation (P3). 
Comparing the regressions in Tables 6 and 7 also indicates 
that the effect of the communications quality variable 
essentially nullifies that of task-oriented leadership (PI). 
For reasons discussed previously, crew cohesiveness, highly 
correlated with crew coordination, was excluded from enter- 
ing the stepwise regression equations. 

Decision com ma nd r evers al is shown by Table 6 to have 
60% of the variance explained by five variables. In decreas- 
ing order of contribution, for the variable order shown, the 
IVs are task-oriented leadership (P2), decision difficulty, 
task-oriented leadership (PI), communications quality (PI), 
and decision participatory leadership (PI). The alternative 
order shown in the hierarchical procedure in Table 7 has no 
less an arbitrary variable order than the stepwise procedure 
of Table 6 and does not contradict the major suggestions 
from Table 6: The behavioral variables of most importance 
are the task-oriented leadership variables for the first 
officer and captain, correlated positively and negatively 
respectively with command reversal, and accounting for about 
28% and 15% increments in variance explained respectively. 
Considering also the 10% increment of variance explained by 
decision difficulty, the suggestion is that a combination of 
low task-oriented captain leader behavior with high task- 
oriented first officer behavior fosters command reversal, 
and that this is particularly so as crews get themselves 
into difficult situations. The other two IVs together 
account for an 8% increment in variance explained; five per- 
cent of which is attributable to the communications quality 
of the captain, also negatively correlated with command 
reversal . 

As mentioned above, decision command reversal (V24) 
correlated significantly (.92) with a more general 
leadership-associated measure of command reversal (V9). Some 
analyses including a case in which V24 was an independent 
and a dependent variable, were repeated with V9 substituted 
for V24 . For example, repetition of the preceding stepwise 
regression, with V9 substituted for V24, produced similar 
results; All the variance explained was attributable to the 
first officers ' and captains ' task-oriented leadership 
behaviors and decision difficulty: about 35%, 11%, and 5% 
increments in multiple R squared respectively. A repetition 
of hierarchical Regression V of Table 7 also produced simi- 
lar results with V9 substituted f or V24 . V24 in conjunction 
with decision efficiency and decision quality explained 46% 
of the variance in safety performance as opposed to 45% when 
V9 was substituted — with 16% contributed by V24 as opposed 
to 9% contributed by V9. The overall evidence is that V9 
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and V24 differ little as currently defined and measured. 

Although it may be appropriate in this study to con- 
sider these two variables as essentially synonymous, such is 
not necessarily recommended for future work - and improve- 
ments in definition may be appropriate. 

A summary suggestion from the above series of regres- 
sion analyses is that major differences for coordination 
versus decisionmaking pathways in the model are associated 
with the effect of command reversal for decisionmaking and 
competency-motivation of the first officer and flight 
engineer for coordination. 

Factor Ana ly sis 

A factor analysis with principal components extraction 
and varimax rotation was performed using BMDP4M (BMDP Sta- 
tistical Software, 1983). The number of factors was limited 
to the number of eigenvalues greater than one. Rotated fac- 
tor loadings for the five orthogonal factors and variance 
explained by each factor are shown in Table 8. 

The highest loadings for fa cto r eng (ranging from .844 
to .923) clearly cluster captain participatory and person- 
oriented leadership variables, and captain communications 
quality variables. Crew cohesiveness , crew coordination, and 
decision efficiency loadings group separately and load on 
factor one (.590 to .641) but also load substantially on 
factors two, three, and four. Task-oriented leadership (PI) 
is clearly grouped with these three crew variables and also 
loads substantially on factor 4. 

Fac to r t wo shows a similar pattern to factor one in 
that the highest loadings clearly cluster first officer par- 
ticipatory and person-oriented leadership variables, and 
first officer communications quality variables . Competency- 
motivation (P2) and task-oriented leadership (P2) group 
together somewhat separately from the above cluster. 

Factor three shows a similar pattern, loading most 
heavily for flight engineer communications quality and the 
competency-motivation variables . 

Fac to r fo ur has its highest loadings on the command 
reversal variables ( . 842 and .727). The only other positive 
loading ( . 619) is for decision difficulty. The important 
negative loadings are for the substantive dependent meas- 
ures , flight safety and decision quality; the intermediate 
performance measures , crew coordination and decision effi- 
ciency ; and task-oriented leadership (PI) . 
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F&Qtor f ive loads most heavily on crew friendliness 
(.574) followed by task-oriented leader behavior of the 
first officer (.390) and captain (.369), and the negative 
loading for the first officer ' s participatory leader 
behavior in decisionmaking (-.312). It is also of some 
interest that crew friendliness loads (.522) on factor one - 
thus at about the same level as does task-oriented leader- 
ship (PI). 

The outcome of the factor analysis tends to confirm the 
model structure suggested by regression analysis ; adding 
knowledge concerning the clustering of communications qual- 
ity variables and other -than- task- oriented leadership vari- 
ables ; and identifying a factor that loads most heavily on 
crew friendliness . Further factor analyses , and regression 
analyses using factors , may be appropriate . 

Grew Di ff erences 

Figure 5, graphically displays the means and SDs for 
the 16 crews on each of the variables of the partial model - 
and also for task-oriented leadership ( P2) . The crew presen- 
tation order is on the basis of ratings on safety perfor- 
mance . Table 9 indicates where each crew landed and the 
amount of fuel on board at landing . 

The following two conversations are presented only as 
examples of communications related to the crew coordination 
process and to decisionmaking . They are from the crews rated 
highest and lowest on safety performance (numbers 13 and 
eight) , and exemplify successful coordination and unsuccess- 
ful decisionmaking , respectively. 


Captain : Don't forget to fly the airplane 
First officer : Yeah , I am (sounding 
slightly defensive) . 

- pause - 

First officer : Everything ' s under control . 

Captain : That ' s your main responsibility . 

First Officer : Yep. 

An exchange like the above occured six times between the 
captain and first officer of crew 13 during this flight leg . 
This particular clarification of responsibility occured 
while both the captain and first officer were copying 
weather . 


Captain : (On approach to LAX) We're going to land 
this way . Tell him to get the fire trucks , 
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we're probably going to smear the nose wheel , 

First officer : (To ATC) We-ah-oould have a problem 
with our nose gear , so be aware of that 
please . 

Captain : ( continues approach) 

First officer : We don't have the green light on our 
nose gear . Ah we ' 11 continue the approach. 

Captain : If that 12K (referring to 12,000 lbs of 
fuel) is realistic , we don ' t need to smear 
this thing . 

This conversation occurred within crew eight , a crew for 
which members appeared to function very independently of 
each other (see their low coordination rating on Figure 5). 
In this conversation the last stated insight of the captain 
received no support from other crewmembers and the aircraft 
was landed unsafely at LAX. 

Concluding Sta te ment 

A start has been made on the development of a model of 
crew coordination and decisionmaking . The inclusion of deci- 
sion efficiency and command reversal as variables in the 
model appear to be useful advances in conceptualizing the 
crew performance process . Some insight was suggested on the 
dynamics of command reversal : Relatively low and high task- 
oriented leader behavior of the captain and first officer 
respectively - especially as the situation becomes difficult 
appears to be the leading impetus to the occurrence of com- 
mand reversal . Task-oriented leadership thus appears to be 
distinguished some from person-oriented or participatory 
leadership . The latter leader behavior variables are shown 
by factor analysis to cluster closely with communications 
quality variables . 

Both regression and factor analyses suggest the impor- 
tant effects of all three crewmember behavioral qualities on 
crew coordination and , through different pathways , on deci- 
sion efficiency . Both analyses also suggest the important 
effects of command reversal in decision pathways - on deci- 
sion efficiency and quality variables , and on safety perfor- 
mance . As outcome variables decision quality is seen to have 
more variance explained (60%) than did safety performance 
(46%). The reason is suggested to be that the defining cri- 
teria for decision quality ratings are more inclusive , con- 
sidering not only safety of flight but also the attainment 
of other mission goals - and were available to raters 
throughout the rating effort . 

Significant decreases in the ratings of leader 
behaviors and communications qualities occurred for both the 
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captain and first officer, for the major problem segment as 
opposed to the pre-problem (take off and cruise to LAX) seg- 
ment. Similar decreases occured for crew coordination, 
cohesiveness, and friendliness - but not for command rever- 
sal and competency-motivation of the first officer and 
flight engineer. It is perhaps significant for command 
reversal that the only leader behavior not rated lower on 
the major problem (as opposed to the pre-problem) segment 
was the first officers' task-oriented leadership. 

One of the five orthogonal factors loaded most heavily 
for crew friendliness. However, the validity of crew 
cohesiveness as a substantial measure as defined and used in 
this study was questioned. 

The rating of videotapes appear to have considerable 
promise in developing crew performance models. A suggested 
improvement on this study is the inclusion of a display of 
systems information, such as airspeed, altitude, fuel 
remaining, etc., adjacent to the video display. Such a 
display should reduce error variance in crew process rat- 
ings, and permit addition to the model of a variable, paral- 
leling decision quality, assessing flight task execution 
quality. 

Considerable reduction in error variance should also be 
realized through refinement of variables and/or their defin- 
ing criteria, through improvements in rater training made 
possible by these videotapes, and through improvements in 
data generation and rating procedures. 


Acknowledgements 

The authors wish to express appreciation to Captains 
Russ Cottle, William A. Dixon, Bill Me Dowell, William P. 
Monan, Harry W. Orlady, and R. W. (Bob) Petersen for their 
enthusiastic and conscientious performances in providing the 
ratings. Also thanks to Informatics employees William Carson 
and Donna Miller for their invaluable assistance with the 
data analysis. 


23.19 



References 


Cooper, G. E., & White, M. D. , & Lauber , J. K. (Eds.). 
(1980). Resource management on the flight deck. 
Proceedings of a NAgA/ Indust ry Wo r kshop . Moffett Field, 
Ca. : NASA Ames Research Center. 

Dixon, W.J., & Brown, M. B. , & Engelman, L. , & Frane, J. W. , 
& Hill, M. A., & Jennrich. R. I, & Toporek, J. D. (Eds.). 
(1983). BMDP Statistical S of tware (rev. ed.). Berkely: 
University of California Press. 

Foushee, H. C. & Manos, K. L. (1981). Information transfer 
within the cockpit : Problems in intraoookpit 
communications. In C. E. Billings & E. S. Cheaney (Eds.) 
Information trans fer problems in t he aviation s ys tem. 
(Report No. NASA TP-1875). Washington, DC: National 
Aeronautics and Space Adminstration . 

Frankel, R. M. , & Beckman, H. B. (1982). Impact: an 
interaction-based method for preserving and analyzing 
clinical transactions. In L.S. Pettegrew (Ed.), 
Explorati on s in provider and patient in teraction (pp. 71- 
85). Humana Inc. 

Goguen, J. Linde, C. , & Murphy, M. (1984) Crew communication 
as a factor in aviation accidents. In E J. Hartzell & S. 
Hart (Eds.), Papers fr om t he 2 0t h An nu al Conference on 
Manual Control, 2, (pp. 217-248). Moffett Field, CA: NASA, 
Ames Research Center. 

Hackman, J. R. & Morris, C. G. (1975). Group tasks, group 
interaction process, and group performance effectiveness: 

A review and proposed integration. In L. Berkowitz (Ed.), 
Advanoss in expe rim ental so c ial ps ychology (Vol. 8, pp 
45-99). New York: Academic Press. 

House, R. J. (1984). Commentary on management research. In 
A. P. Brief (Ed.), Produc tivi ty r ese arch in t he be hav iora l 
and soci al scienc es (pp. 268-282). New York: Praeger. 

House, R. J. , & Baetz, M. L. (1979). Leadership: Some 
empirical generalizations and new research directions. 
Research a nd organi zat ional behav ior (Vol. 1, pp. 341- 
423). JAI Press Inc. 

Kerr, S. (1984). Leadership and participation. In A. P. 

Brief (Ed.), Produc ti vity r es ear ch in th e beha vi o ra l a nd 
social sciences (pp. 229-251). New York: Praeger. 

Landy , F . J. , & Farr , J . L. ( 1980) . Performance rating . 


23.20 



Psyc hol ogical Bull et in . £7, 72-107. 

Landy, F. J. , & Farr, J. L. (1983). The measurement of work 
perf Qrmanee • New York: Academic Press. 

Murphy, M. R. (1977). Coordinated crew performance in 

commercial aircraft operations. Pr oc e ed ings of the 21 st 
Human Factors Society Annual Meeting, 416-420. 

Murphy, M. R. (1980). Analysis of eighty-four commercial 
aviation incidents: implications for a resource management 
approach to crew training. I960 Proceed in gs Annua l 
Reliablility and Maintainability Symposi um . 298-306. 

National Transportation Safety Board, Special Study. (1976). 
Flight erew co ordinati on pro ce dures in air carrier 
instrument landing system approach acciden ts (Report No. 
NTSB-AAS-76-5) . Washington, DC: National Transportation 
and Safety Board. 

Ruffell Smith, H.P. (1979). A si mu lator study of the 

interaction of pilot wor kl oad with errors, vigilance , and 
dec is ions (Report No. NASA TM-78482). Washington DC: 
National Aeronautics and Space Adminstration . 


23.21 



Table 1 


Concept-Variable Relationships 


Concept 

♦Referent 

Var. No. 

♦♦Instrument 

a. Task-Oriented Leadership 

PI ,P2 

1,3 

CP 

b. Person-Oriented Leadership 

PI ,P2 

2,4 

CP 

c. Participatory Leadership 

PI ,P2 

5,6 

CP 



19,20 

DM 

d . Competency-Motivation 

P2,P3 

7,8 

CP 

e. Command Reversal 

P1-P2 

9 

CP 



24 

DM 

f. Crew Cohesiveness 

Crew 

10 

CP 

g. Crew Coordination 

Crew-Task 

11 

CP 

h. Crew Friendliness 

Crew 

12 

CP 

i. Communications Quality 

PI ,P2,P3 

13,14,15 

CP 



21,22,23 

DM 

j . Decision Difficulty 

Situation 

16 

DM 

k. Decision Quality 

Crew Outcome 

17 

DM 

1. Decision Efficiency 

Crew-Task 

18 

DM 

m. Safety Performance 

Crew Outcome 

25 

SP 


*P1 = Captain, P2 = First Officer, P3 = Flight Engineer 
**CP = Crew Process, DM = Decision Making, SP = Safety Performance 


Table 2 


Variable Reliabilities Determined by Spearman-Brown Formula for 
Multiple r 


No. 

r 

No. 

r 

No. 

r 

No. 

r 

No. 

r 

1 

.91 

6 

.87 

11 

.96 

16 

.88 

21 

.89 

2 

.93 

7 

.93 

12 

.90 

17 

.93 

22 

.86 

3 

.78 

8 

.96 

13 

.94 

18 

.94 

23 

.92 

4 

.90 

9 

.96 

14 

.92 

19 

.88 

24 

.95 

5 

.92 

10 

.96 

15 

.96 

20 

.84 

25 

.99 
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Table 3 


Changes in Crew Process Variable Means Across Segments 


No. 

Name 

Means 

1 

for Segments 
2 3 

'Decrease 
1 to 2 1 to 3 

Overall 
Mean SD 

1 

Task-Oriented 
Leadership (PI) 

5.08 

4.71 

4.56 

♦ .37 

♦ ♦ . 52 

4.79 

1.28 

2 

Person-Oriented 
Leadership (PI) 

4.48 

3.93 

3.91 

♦ ♦ . 55 

♦ ♦ .57 

4.11 

1.32 

3 

Task-Oriented 
Leadership (P2) 

4.97 

4.91 

4.86 

ns 

ns 

4.92 

1.01 

4 

Person-Oriented 
Leadership (P2) 

4.72 

4.34 

4.30 

♦ .38 

♦ ♦ .42 

4.46 

1.09 

5 

Participatory 
Leadership (PI) 

4.71 

3.91 

3.84 

♦ ♦ . 80 

♦ ♦ . 87 

4.16 

1.47 

6 

Participatory 
Leadership (P2) 

4.91 

4.13 

4.38 

*♦ . 78 

♦ * . 53 

4.48 

1.18 

7 

Competency- 
Motivation (P2) 

5.10 

4.87 

4.98 

ns 

ns 

4.99 

1.29 

8 

Competency- 
Motivation (P3) 

4.50 

4.15 

4.42 

ns 

ns 

4.36 

1.34 

9 

Command 

Reversal 

3.60 

3.86 

3.81 

ns 

ns 

3.76 

1.59 

10 

Crew 

Cohesiveness 

4.56 

3.89 

4.02 

♦ ♦.67 

♦ .54 

4.16 

1.56 

11 

Crew 

Coordination 

4.49 

3.86 

3.83 

+♦ . 63 

♦ ♦.66 

4.07 

1.57 

12 

Crew 

Friendliness 

4.83 

4.35 

4.49 

♦ ♦ . 48 

♦ .34 

4.55 

1.03 

13 

Communications 
Quality (PI) 

4.82 

4.17 

4.06 

♦ ♦ . 65 

♦ ♦.76 

4.36 

1.51 

14 

Communications 
Quality (P2) 

4.97 

4.53 

4.53 

♦ .44 

+ .44 

4.68 

1.23 

15 

Communications 
Quality (P3) 

4.53 

4.14 

4.44 

ns 

ns 

4.37 

1.19 


♦Difference significant (p<.05) 

♦♦Difference significant (p<.01) 

'Differences between segments 2 and 3 means were not significant 

(p>.10) 

Table 4 


Means of Decisionmaking Variables for the Eight Decisions 


No. 

Name 

1 

Mean 

2 

Mean 

Decis 

3 

Mean 

ion Number 
4 5 

Mean Mean 

6 

Mean 

7 

Mean 

8 

Mean 

Overall 
Mean SD 

16 

Decision 

Difficulty 

3.11 

2.21 

4.23 

3.24 

3.89 

3.41 

2.49 

2.54 

3.16 

1.45 

17 

Decision 

Quality 

4.54 

6.11 

3.97 

5.10 

4.71 

4.77 

5.65 

4.29 

4.86 

1.65 

18 

Decision 

Efficiency 

3.84 

4.52 

3.18 

3.32 

3.18 

3.43 

4.38 

3.48 

3.65 

1.72 

19 

Decision 

Participatory 

Leadership(Pl) 

4.65 

4.05 

3.78 

3.75 

3.77 

3.55 

3.63 

3.53 

3.86 

1.42 

20 

Decision 
Participatory 
Leadership (P2) 

4.65 

4.24 

4.26 

4.11 

4.20 

3.75 

3.70 

4.27 

4.20 

1.25 

21 

Decision 
Communications 
Quality (PI) 

4.52 

4.50 

3.85 

3.96 

4.01 

3.97 

3.97 

3.77 

4.08 

1.42 

22 

Decision 
Communications 
Quality (P2) 

4.83 

4.44 

4.23 

4.18 

4.39 

3.88 

4.22 

4.36 

4.34 

1.22 

23 

Decision 
Communications 
Quality (P3) 

4.26 

4.19 

4.00 

3.74 

3.81 

4.04 

3.92 

4.07 

4.01 

1.21 

24 

Decision 

Command 

Reversal 

3.53 

3.36 

3.58 

3.78 

3.69 

3.05 

2.82 

3.54 

3.45 

1.56 


3,23 



Table 5 


ORIGINAL PAGE IS 

Correlation Matrix OF POOR QUALITY 




1 

2 

3 

4 

5 

6 

7 

1 

Task-Oriented Leadership (PI) 

1 . 0000 

0. 4768 

0.2226 

0.1267 

0. 4425 

0.0883 

0.2476 

2 

Person-Oriented Leadership (PI) 

0.4788 

1 . 0000 

0.2164 

0. 1652 

0.8170 

0.1622 

0.3990 

3 

Task-Oriented Leadership (P2) 

0.2226 

0.2164 

1 . 0000 

0.4983 

0. 1629 

0.6470 

0.6412 

4 

Person-Oriented Leadership (P2) 

0. 1267 

0. 1652 

0. 4983 

1 . 0000 

0. 1857 

0.6880 

0.5766 

5 

Participatory Leadership (PI) 

0.4425 

0.8170 

0. 1629 

0.1857 

1 . 0000 

0 . 2756 

0.4206 

6 

Participatory Leadership (P2) 

0.0883 

0. 1622 

0.6470 

0.6880 

0.2756 

1 . 0000 

0.6969 

7 

Competency-Motivation (P2) 

0.2476 

0.3990 

0.6412 

0.5766" 

0.4206 

0.6969 

1 . 0000 

8 

Competency-Motivation (P3) 

0.231 1 

0.2353 

0.5283 

0.4028 

0.2733 

0.4666 

0.6182 

9 

Command Reversal 

-0.3241 

-0.1315 

0.5005 

0.2427 

-0.0368 

0.4061 

0.2668 

1 0 

Crew Cohesiveness 

0.5528 

0.6368 

0.4442 

0.4076 

0.6549 

0.4730 

0.7436 

1 1 

Crew Coordination 

0.5453 

0.6145 

0.4031 

0.3523 

0.6693 

0.4490 

0.6816 

12 

Crew Friendliness 

0.5144 

0.6676 

0.4290 

0.5288 

0.5224 

0. 3034 

0.4509 

13 Communications Quality (PI) 

0.5785 

0.7998 

0. 1434 

0. 1581 

0.7992 

0.2367 

0.4499 

1 4 

Communications Quality (P2) 

0.2790 

0. 3028 

0.6137 

0.6432 

0.3012 

0.7007 

0.8098 

15 Communications Quality (P3) 

0.1310 

0. 1635 

0.4760 

0.3340 

0. 1 988 

0.3686 

0.4803 

1 6 

Decision Difficulty 

-0.2881 

-0.0472 

-0. 1302 

-0.0734 

0.0171 

-0.0074 

-0.0581 

1 7 

Decision Quality 

0.3888 

0.2913 

0. 1722 

0.1186 

0. 3210 

0.2309 

0.3343 

1 8 

Decision Efficiency 

0.5639 

0.5157 

0.2518 

0.2533 

0.5957 

0.3701 

0 . 5638 

19 

Decision Participatory Leadership (PI) 

0.3334 

0.6917 

0. 1798 

0.2087 

0.8276 

0.3105 

0.3989 

20 

Decision Participatory Leadership (P2) 

-0.0892 

0.0716 

0.5180 

0.6101 

0.2010 

0.7667 

0 .5593 

21 

Decision Communications Quality (PI ) 

0.4759 

0.7191 

0.1516 

0.1716 

0.7559 

0.3080 

0.4683 

22 

Decision Communications Quality (P2) 

0.0263 

0. 2107 

0.5025 

0.5399 

0.2720 

0.7214 

0.6524 

23 

Decision Communications Quality (P3) 

-0.0085 

0. 1907 

0.3125 

0.2250 

0.2330 

0.3393 

0.4156 

24 

Decision Command Reversal 

-0 . 3353 

-0.0992 

0.4257 

0. 1 760 

-0.0242 

0.2978 

0.2035 

25 

Safety Performance 

0. 4094 

0.2673 

0.0935 

0.1517 

0.2988 

0. 1998 

0.2358 



8 

9 

10 

1 1 

12 

13 

1 4 

15 

i 6 

1 

0.2311 

-0.324 1 

0.5528 

0.5453 

0.5144 

0.5785 

0.2730 

0.1310 

-0.2881 

2 

0.2353 

-0. 1315 

0.6368 

0.6145 

0.6676 

0.7998 

0 . 3028 

0. 1 635 

-0.0472 

3 

0.5283 

0.5005 

0.4442 

0.4031 

0 . 4290 

0. 1434 

0.6137 

0. 4760 

-0. 1302 

4 

0.4028 

0.2427 

0 . 4076 

0.3523 

0.5288 

0. 1581 

0.6432 

0.3340 

-0.0734 

5 

0.2783 

-0.0368 

0.6549 

0.6693 

0.5224 

0.7992 

0.3012 

0. 1930 

0.0171 

6 

0.4666 

0. 4061 

0 . 4730 

0. 4490 

0.3034 

0.2367 

0.7007 

0.3686 

-0.0074 

7 

0.6182 

0.2668 

0.7436 

0.6816 

0.4509 

0.4493 

0.8098 

0.4803 

-0.0581 

8 

1 . 0000 

0. 1852 

0.6863 

0. 6G74 

0.3927 

0.2925 

0.5268 

0. 8686 

-0. 1803 

9 

0. 1852 

1 . 0000 

-0. 1056 

-0.0744 

-0.0973 

-0.2549 

0. 1687 

0.2174 

0.2703 

10 

0.6868 

-0. 1056 

1 . 0000 

0.9337 

0.6295 

0,7539 

0,6731 

0,5342 

-0.24)0 

1 1 

0.6674 

-0.0744 

0.9337 

1 .0000 

0 .5603 

0.7427 

0.6035 

0.5090 

-0.2232 

12 

0.3927 

-0.0973 

0.6295 

0.5603 

1 .0000 

0.5777 

0.5048 

0.3409 

-0.2322 

13 

0.2925 

-0.2549 

0.7539 

0.7427 

0.5777 

1 . 0000 

0 . 4069 

0. 1637 

-0.0831 

1 4 

0.5268 

0. 1687 

0.6731 

0.6035 

0 . 5048 

0. 4069 

1 . 0000 

0.4035 

-0. 1853 

15 

0.8686 

0.2174 

0.5342 

0.5090 

0.3409 

0.1687 

0.4085 

1 . 0000 

-0.0999 

1 6 

-0. 1803 

0 .2703 

-0.2410 

-0.2232 

-0.2322 

-0.0831 

-0. 1853 

-0.0999 

1 . 0000 

17 

0.3928 

-0. 1437 

0.4780 

0.5461 

0.2237 

0.3488 

0.2307 

0.3937 

-0.2500 

18 

0.5054 

-0. 1 928 

0 .7727 

0.8206 

0. 4375 

0.6528 

0. 4606 

0-4115 

-0.2305 

1 9 

0.2867 

0.0970 

0.5673 

0.5362 

0 . 4385 

0.6437 

0.3062 

0.2471 

0.0185 

20 

0.4559 

0. 4838 

0.3379 

0.2861 

0.1641 

0.1188 

0.5573 

0.4000 

0.0772 

21 

0.2915 

-0.1446 

0.6917 

0.6836 

0.4168 

0.7938 

0.4040 

0. 1928 

-0. 1462 

22 

0.4775 

0 . 3584 

0. 4743 

0.4160 

0.2638 

0.2916 

0.7287 

0.4394 

-0.0189 

23 

0.6974 

0.254 i 

0.4362 

0.4025 

0.1611 

0. 1 368 

0.2508 

0.7797 

0.0571 

24 

0. 1534 

0.9198 

-0. 1259 

-0. 1286 

-0.1112 

-0.2500 

0.0934 

0. 1851 

0.3729 

25 

0.2707 

-0. 3049 

0.4166 

0. 4589 

0.2828 

0.2891 

0.1591 

0.2059 

-0.3415 



17 

18 

19 

20 

21 

22 

23 

24 

25 

1 

0.3G88 

0.5639 

0.3334 

-0.0892 

0.4759 

0.0263 

-0.0085 

-0.3853 

0.4094 

2 

0.2913 

0.5157 

0. 6917 

0.0716 

0.7191 

0.2107 

0. 1 907 

-0.0992 

0.2673 

3 

0. 1722 

0.2518 

0. 1 798 

0.5180 

0.1516 

0.5025 

0.3125 

0.4257 

0.0935 

4 

0.1186 

0.2533 

0.2087 

0.6101 

0.1716 

0.5399 

0.2250 

0. 1760 

0.1517 

5 

0.3210 

0.5957 

0,8276 

0.2010 

0.7559 

0.2720 

0.2330 

-0.0242 

0.2988 

6 

0.2309 

0.3701 

0.3105 

0.7667 

0.3080 

0.7214 

0,3393 

0.2978 

0. 1998 

7 

0.3343 

0.5638 

0,3989 

0.5593 

0.4683 

0.6524 

0.4156 

0.2035 

0.2358 

8 

0.3928 

0.5054 

0.2867 

0.4559 

0.2915 

0. 4775 

0.6974 

0. 1534 

0.2707 

9 

-0. 1437 

-0. 1928 

0.0970 

0.4838 

-0.1446 

0.3584 

0,2541 

0.9198 

-0.3049 

10 

0.4780 

0.7727 

0.5673 

0.3379 

0.6917 

0.4748 

0.4362 

-0. 1259 

0.4166 

1 1 

0.5461 

0.8206 

0.5362 

0.2861 

0.6836 

0.4160 

0.4025 

-0.1286 

0.4589 

12 

0.2237 

0, 4375 

0.4385 

0.1641 

0. 4168 

0.2638 

0.1611 

-0.1112 

0.2828 

13 

0.3488 

0.6528 

0.6437 

0.1183 

0.7938 

0.2916 

0. 1368 

-0.2500 

0.2891 

1 4 

0.2307 

0.4606 

0.3062 

~0.5573 

0. 4040 

0.7287 

0.2503 

0.0934 

0. 1591 

15 

0.3937 

0.4115 

0.2471 

0. 4000 

0. 1928 

0.4394 

0.7797 

0. 1851 

0.2059 

16 

-0.2500 

-0.2305 

0.0185 

0.0772 

-0, 1462 

-0.0189 

0.0571 

0.3729 

-0.3415 

1 7 

1 . 0000 

0.7533 

0 .3305 

0. 1409 

0. 4453 

0.2104 

0.3852 

-0.2444 

0.5795 

18 

0.7533 

1 . 0000 

0.5490 

0.2096 

0.6876 

0.3484 

0.4180 

-0.2963 

0.6204 

1 9 

0.3305 

0 . 5490 

1 . 0000 

0.4071 

0.804 1 

0.4429 

0. 4047 

0. 1368 

0.2742 

20 

0, 1409 

0.2096 

0.4071 

1 . 0000 

0,3301 

0.8384 

0.4975 

0.4817 

0,1215 

21 

0.4453 

0.6876 

0 . 804 1 

0.3301 

1 . 0000 

0.4762 

0.3554 

-0.1437 

0.4184 

22 

0.2104 

0.3484 

0.4429 

0.8384 

0. 4762 

1 . 0000 

0.5020 

0.3382 

0.0959 

23 

0.3852 

0.4180 

0.4047 

0. 4975 

0.3554 

0.5020 

1 . 0000 

0.2871 

0.2649 

24 

-0.2444 

-0.2963 

0. 1363 

0.4817 

-0. 1437 

0.3382 

0.2871 

1 . 0000 

-0.3979 

25 

0.5795 

0.6204 

0.2742 

0.1215 

0.4184 

0.0959 

0.2649 

-0.3979 

1 . 0000 


Correlations at or above .267 are significant for n=90 df . 
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Table 6 


Stepwise Regression Analysis Summaries 


Step 

Variable Entered 

Variable 

Removed 

Multiple 
R RSQ 

Change 
in RSQ 

I DECISION COMMAND REVERSAL (V24) 

1 13. Communications Quality (Pi) 


.2500 .0625 

.0625 

2 

1 .Task-Oriented Leadership (PI) 


.3867 .1496 

.0871 

3 


13 

.3853 .1485 

-.0011 

4 

3. Task-Oriented Leadership (P2) 


.6509 .4237 

.2752 

5 

16. Decision Difficulty 


.7216 .5207 

.0970 

6 

19. Decision Participatory Leadership (PI) 


.7441 .5537 

.0330 

7 

13. Communications Quality (PI) 


.7774 .6043 

.0506 

II CREW COORDINATION (VI 1) 

1 1 .Task-Oriented Leadership (PI) 


.5453 .2973 

.2973 

2 

21. Decision Communications Quality (PI) 


.7279 .5299 

.2325 

3 

7 . Competency-Motivation (P2) 


.8313 .6910 

.1611 

4 

8 . Competency-Motivation (P3) 


.8794 .7734 

.0824 

5 

24. Decision Command Reversal 


.8857 .7844 

.0110 

6 

13 . Communications Quality (PI) 


.9053 .8196 

.0352 

7 


24 

.9024 .8144 

-.0053 

8 


21 

.8991 .8083 

-.0060 

III 

1 

DECISION EFFICIENCY (V18) 

1 . Task-Oriented Leadership (PI) 


.5639 .3180 

.3180 

2 

21. Decision Communications Quality (PI) 


.7384 .5452 

.2272 

3 

11. Crew Coordination 


.8461 .7158 

.1706 

4 

24. Decision Command Reversal 


.8591 .7380 

.0221 

5 


1 

.8577 .7357 

-.0023 

6 

23. Decision Communications Quality (P3) 


.8720 .7604 

.0247 

7 

9. Command Reversal 


.8849 .7831 

.0228 

IV DECISION QUALITY (VI 7) 

1 1 . Task-Oriented Leadership (PI) 


.3888 .1512 

.1512 

2 

21. Decision Communications Quality (PI) 


.4886 .2387 

.0875 

3 

11. Crew Coordination 


.5627 .3167 

.0779 

4 


21 

.5568 .3101 

-.0066 

5 


1 

.5461 .2983 

-.0118 

6 

24. Decision Command Reversal 


.5737 .3291 

.0308 

7 

18. Decision Efficiency 


.7638 .5834 

.2543 

8 


11 

.7536 .5680 

-.0154 

9 


24 

.7533 .5675 

- . 0005 

10 

13 . Communications Quality (PI) 


.7766 .6031 

.0356 

V SAFETY PERFORMANCE (V25) 

1 1 . Task-Oriented Leadership (PI) 


.4094 .1676 

.1676 

2 

21. Decision Communications Quality (PI) 


.4819 .2323 

.0646 

3 

24. Decision Command Reversal 


.5539 .3068 

.0745 

4 


1 

.5400 .2915 

-.0152 

5 

11. Crew Coordination 


.5841 .3412 

.0496 

6 


21 

.5721 .3274 

-.0138 

7 

18. Decision Efficiency 


.6610 .4369 

.1095 

8 


11 

.6597 .4352 

-.0017 

9 

17. Decision Quality 


.6795 .4617 

.0266 
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Table 7 


Hierarchical Regression Analysis Summaries 



Variable Entered 

Multiple 
R RSQ 

Change 
in RSQ 

I 

DECISION COMMAND REVERSAL (V24) 
16. Decision Difficulty 

.3729 .1391 

.1391 


19. Decision Participatory Leadership (PI) 

.3949 

.1559 

.0169 


3. Task-Oriented Leadership (P2) 

.6080 .3697 

.2137 


13. Communications Quality (PI) 

.7323 .5363 

.1660 


1 .Task-Oriented Leadership (PI) 

.7774 .6043 

.0680 

II 

CREW COORDINATION (VI 1) 

13. Communications Quality (PI) 

.7427 .5516 

.5516 


1. Task-Oriented Leadership (PI) 

.7561 

.5717 

.0201 


7 . Competency-Motivation (P2) 

.8515 .7251 

.1534 


8 . Competency-Motivation (P3) 

.8991 

.8083 

.0833 

III 

DECISION EFFICIENCY (V18) 

23. Decision Communications Quality (P3) 

.4180 .1747 

.1747 


21. Decision Communications Quality (PI) 

.7122 .5073 

.3326 


24. Decision Command Reversal 

.7680 .5898 

.0825 


9 . Command Reversal 

.8012 .6419 

.0520 


11. Crew Coordination 

.8849 .7831 

.1412 

IV 

DECISION QUALITY (V17) 

13. Communications Quality (PI) 

.3488 .1216 

.1216 


18. Decision Efficiency 

.7766 .6031 

.4814 

V 

SAFETY PERFORMANCE (V25) 

24. Decision Command Reversal 

.3979 

.1583 

.1583 


18. Decision Efficiency 

.6597 .4352 

.2769 


17. Decision Quality 

.6795 .4617 

.0266 
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Table 8 


Factor Analysis Using Principal Components and Varimax Rotation 



VARIABLES 

FACTOR 

FACTOR 

FACTOR 

FACTOR 

FACTOR 



1 

2 

3 

4 

5 

5. 

Participatory Leadership (PI) 

0.923 

0. 

0. 

0. 

0. 

2. 

Person-Oriented Leadership (PI) 

0.888 

0. 

0. 

0. 

0. 

13. 

Communications Quality (PI) 

0.882 

0. 

0. 

0. 

0. 

21. 

Decision Communications Quality (PI) 

0.846 

0. 

0. 

0. 

0. 

19. 

Decision Participatory Leadership (PI) 

0.844 

0. 

0. 

0. 

0. 

10. 

Crew Cohesiveness 

0.641 

0. 393 

0.409 

-*0.306 

0.252 

11. 

Crew Coordination 

0.639 

0.333 

0.4 30 

-0.327 

0. 

18. 

Decision Efficiency 

0.590 

0. 

0.406 

-0.517 

0. 

1 . 

Task-Oriented Leadership (PI) 

0. 505 

0. 

0. 

-0.479 

0.369 

6. 

Participatory Leadership (P2) 

0. 

0. 882 

0. 

0. 

0. 

20. 

Decision Participatory Leadership (P2) 

0.. 

0.812 

0. 

0. 

-0.312 

14. 

Communications Quality (P2) 

0. 

0.812 

0. 

0. 

0. 

22. 

Decision Communications Quality (P2) 

0. 

0.801 

0. 

0. 

0. 

4. 

Person-Oriented Leadership (P2) 

0, 

0. 794 

0. 

0. 

0. 

7. 

Competency-Motivation (P2) 

0*356 

0.711 

0.324 

0. 

0. 

3. 

Task-Oriented Leadership (P2) 

0, 

0. 657 

0.341 

0. 

0.390 

15. 

Communications Quality (P3) 

0. 

0.255 

0.889 

0. 

0. 

23. 

Decision Communications Quality (P3) 

0. 

0. 

0.836 

0. 

0. 

8. 

Competency-Motivation (P3) 

0* 

0.368 

0.814 

0. 

0. 

24. 

Decision Command Reversal 

0. 

0.315 

0. 

0.842 

0. 

9. 

Command Reversal 

0. 

0. 408 

0. 

0.727 

0. 

25. 

Safety Performance 

0.254 

0 . 

0.258 

-0.671 

0. 

16. 

Decision Difficulty 

0. 

0. 

0. 

0.619 

0, 

17. 

Decision Quality 

0.300 

0. 

0.482 

-0.528 

0. 

12. 

Crew Friendliness 

0.525 

0.308 

0. 

0. 

0.574 


VARIANCE EXPLAINED 

6.083 

5 . 393 

3.513 

3 . 304 

1.483 


Loadings less than 0.2500 have been replaced by zero 


Table 9 


Airport of Landing and Fuel Remaining at Landing 


Crew 

No. 

Airport 

♦Fuel 

Klbs 

Crew 

No. 

Airport 

♦Fuel 

Klbs 

1 

PMD 

6.2 

9 

PMD 

2.9 

2 

PMD 

7.9 

10 

ONT 

2.0 

3 

ONT 

3.5 

11 

PMD 

7.5 

4 

ONT 

4.6 

12 

PMD 

8.9 

5 

ONT 

3.4 

13 

PMD 

7.7 

6 

PMD 

6.0 

14 

PMD 

2.8 

7 

ONT 

4.0 

15 

PMD 

9.8 

8 

LAX 

8.8 

16 

PMD 

5.4 


* Accuracy of fuel remaining = +/- 105? 
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SEGMENT 1 

SEGMENT 2 

SEGMENT 3 

(PRE-PROBLEM) 

(MAJOR PROBLEM) 

(SECONDARY PROBLEM) 


Fig. 3 Videotape Stopping Points for Decisionmaking 
and Crew Process Ratings 


THE DECISION PROCESS WAS VERY EFFICIENT 

STRONGLY DISAGREE 1 2 3 4 5 6 7 STRONGLY AGREE 

(ALL SIGNIFICANT INFORMATION WAS ACQUIRED AT AN OPPORTUNE TIME, 
ADEQUATELY EVALUATED, AND APPROPRIATELY UTILIZED) 

COMMENT: 


Fig. 4 Rating Scale Example 
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Crew 

No. 


13 
2 

12 

15 

11 

8 

18 

4 

14 
0 
7 
3 

5 
1 

10 

a 


Task -Oriented 

Comm. 

Task-Oriented 

Decision 

Decision 

Crew 

Deolsion 

Decision 

Safety 

Leadership 

Quality (Pi) 

Leadership 

Comm. 

Command 

Coord. 

Ef f ioienoy 

Quality 

Performance 
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Quality (Pi) 
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Fig. 5 Means and Sds of Sixteen Crews on Variables of Model - Descending 
Order of Performance on V25 (Safety Performance} 


23.30 




Appendix A 

Scale and Criterion Statements for Variables - Ordered by Concepts 

a. Task-Oriented Leadership/Variables 1 and 3: 

The CAPTAIN’S (FIRST OFFICER’S) leader behavior was highly task - 
oriented . (His behavior was highly concerned with establishing goals, 
clarifying responsibilities, defining ^subordinate roles and task 
requirements, coaching subordinates and providing task related feed- 
back and evaluation) 

b. Person-Oriented Leadership/Variables 2 and 4: 

The CAPTAIN’S (FIRST OFFICER’S) leader behavior was highly 
sociomotively - oriented . (His behavior was highly concerned with 
establishing and maintaining positive crewmember relationships, pro- 
viding psychological support, enabling feelings of satisfaction, mak- 
ing tasks interesting or enjoyable) 

c. Participatory Leadership/Variables 5 and 6: 

The CAPTAIN’S (FIRST OFFICER’S) leader behavior was highly participa- 
tive in regard to supervision / resources management . (His behavior 
was highly concerned with encouraging subordinates to make sugges- 
tions regarding accomplishment of tasks, to independently analyze 
problems, to give feedback, and to question the leader) 

Participatory Leadership/Variables 19 and 20: 

The CAPTAIN’S (FIRST OFFICER’S) leader behavior was highly participa- 
tive in regard to decisionmaking . (His behavior was highly concerned 
with ensuring that all crewmembers for whom a decision was relevant 
had a chance to influence that decision -i.e. those crewmembers who 
had significant and pertinent information, responsibility to imple- 
ment the decision, or significant ego involvement for other reasons) 

d. Competency-Motivation/Variables 7 and 8: 

The FIRST OFFICER (FLIGHT ENGINEER) exhibited high competence and 
willingness to be responsible with respect to fulfilling the require- 
ments of his position . (appeared highly knowledgeable, skillful, and 
motivated in his behavior and task performance) 

e. Command Reversal/Variable 9: 

The FIRST OFFICER performed much of what would generally be the 

captain ’ s leadership function . (whether due to acquiescence of the 
captain or dominance of the first officer) 

Command Reversal/Variable 24: 

The FIRST OFFICER performed much or what would generally be the 

captain ’s decisionmaking function . (same criterion statement) 

f. Crew Cohesiveness/Variable 10: 

The CREW functioned in a highly cohesive manner . (showed high crew 
solidarity or harmony; i.e. appeared well integrated into a unit) 
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g. Crew Coordination/Variable 11: 

ACTIVITIES OF CREWMEMBERS were well coordinated to task and situation 
demands. (individual crewmember knowledge and skills were allocated 
in an effective and timely manner to meet task and situation demands) 

h. Crew Friendliness/Variable 12: 

The CREWMEMBERS related in a highly friendly manner . (interactions 
were most usually accompanied by verbal and/or non-verbal signs of 
friendliness, or warmth) 

i . Communications Quality /Variables 13,14,15: 

The CAPTAIN (FIRST OFFICER, FLIGHT ENGINEER) exhibited ver£ good 

within - crew communications . (highly hearable, understandable, 
appropriate - in style and content, accurate and timely messages; 
good listener who made effort to understand; achieved reciprocal 
indication that understanding was reached) 

Communications Quality /Variables 21, 22, 23: 

The CAPTAIN (FIRST OFFICER , FLIGHT ENGINEER) exhibited ver£ good 

within - crew communications in regard to decisionmaking . (same cri- 
terion statement) 

j. Decision Difficulty /Variable 16: 

The DECISION was a very difficult one . (involved complex, interact- 
ing operational factors - some of which may have been contingent on 
uncertain future events; or conflicting goals of safety, operational 
efficiency, company and/or ATC requirements - or preferences) 

k. Decision Quality /Variable 17: 

The CHOICE (or outcome of the decision process) was most appropriate . 
(the best choice considering safety of flight and/or the attainment 
of other mission goals) 

l. Decision Efficiency /Variable 18: 

The DECISION PROCESS was very efficient . (all significant informa- 
tion was acquired at an opportune time, adequately evaluated, and 
appropriately utilized) 

m. Safety Perf ormance/Variable 25: 

The LEVEL OF SAFETY achieved through this crew ’s performance , consid- 
ering the major scenario problem and any **special circumstances (s) , 
was ver Y high . (based on: safety of approach (es) to LAX; the airport 
of landing - considering differential risks of LAX,ONT,and PMD; fuel- 
on-board at touchdown - considering go-around and go-to-another- 
alternate fuel requirements, as well as additional fuel for a reason- 
able margin of safety) 

*The word " subordinate 11 was used in referring to "other crewmembers" only 
in reference to the captain’s leadership role. The latter phrase, or a 
derivative, was used throughout in reference to the first officer’s 
leadership role. 

**See Appendix B for a discussion of these special circumstances. 
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Appendix B 

Data Quality __________ 

For many reasons scenario conditions or (contextual events) were 
not entirely consistent over the 16 crews for these long data runs. 
Simulator failures, crew generated events, experimenter team errors, 
planned interventions to reduce possibilities of crashes (for example, 
from remaining too long in the LAX area) contributed to such incon- 
sistencies. An advantage of ratings are that contextual effects can be 
taken into consideration. During the rating procedure all such incon- 
sistencies were identified and discussed with the raters. For the safety 
performance ratings such inconsistencies were documented, by crew, on 
fact sheets. 

Simulator problems included stabilizer chatter that led crew one to 
assume a runaway stabilizer and declare and emergency prior to arrival 
at LAX. This necessitated clearing their flight to approach LAX with 
17,500 lbs of fuel on board. Crews three, seven, and nine were cleared 
for but did not execute the enroute hold prior to fuel levels reaching 
14,000 lbs, and their subsequent clearance to LAX. Although varying 
time-in-hold was planned (to compensate for usual crew differences in 
fuel burn) , simulator burn rates were determined to be high for these 
flight segments. A few crews were given runway visual ranges that were 
below the minimum 2400 ft. on their second approach to LAX. Although 
the crews could and some did, legally continue their approach across the 
outer marker by declaring an emergency, bias toward early interruption 
of approach could be present for some crews. Finally, the flight 
engineer of crew two had some prior knowledge of the major scenario 
problem. All experimenters and raters agreed however that his role play 
as a naive crewmember was successful and should result in little bias 
for that crew’s performance. 
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